In the world of virtualization infrastructure, the term, “live migration” refers to the migration of a virtual machine (VM) from a source host computer to a destination host computer. Each host computer is a physical machine that may reside in a common datacenter or distinct datacenters. On each host, virtualization software includes hardware resource management software, which allocates physical resources to running VMs on the host and emulation software which provide instances of virtual hardware devices, such as storage devices, network devices, etc., that are interacted with by the guest system software, i.e., the software executing “within” each VM. Virtualization software running on each host also cooperates to perform the live migration.
Exemplary implementations of live migration of VMs are described in detail in U.S. Pat. No. 7,484,208 which issued on Jan. 27, 2009, and U.S. Pat. No. 7,680,919, which issued on Mar. 16, 2010. These two patents are expressly incorporated by reference herein. However, in general terms, one key aspect of performing live migration is copying the state of the VM from the source host to the destination host in a manner that allows minimal or insignificant disruption of the VM's execution at the time of the transfer from the source to the destination. A challenging component of this state information to be transferred is the contents of the guest physical memory. A VM's guest physical memory comprise those pages of machine memory (i.e., actual physical memory residing in the host) that are mapped or allocated to the VM being migrated. The guest physical memory address space is treated by the guest system software (e.g., the guest operating system) as actual physical memory, but is of course mapped by the virtualization software to physical pages of machine memory. The main reason it is challenging to transfer the guest physical memory to the destination computer in a live migration is because the VM is allowed to continue to execute during the transfer, and therefore continues to update the guest physical memory as the guest physical memory is being copied or transferred.
To copy guest physical memory to a destination host while the VM is executing, an iterative pre-copy scheme may be used, as described in detail in the patents incorporated by reference above. In general terms, the guest physical memory pages are iteratively copied to the destination host prior to execution of the migrating VM on the destination host. This involves multiple copying operations beginning with copying the entire contents of the guest physical memory to the destination host, then repeatedly copying the pages dirtied (i.e., written to by the VM) since the previous copy operation. Provided the bandwidth for transmitting the copied pages between the source and destination hosts exceed changes to the guest physical memory pages caused by the VM, the pre-copy process will eventually converge to a very small set of pages that can then be successfully copied to the destination host along with other VM state information after stunning (pausing execution of) the VM on the source host so the VM can then be resumed on the destination host with minimal or insignificant interruption.
In some situations, e.g., in cases where the pre-copy technique described above cannot converge, the execution of the VM is transferred to the destination host even before all of the guest physical memory pages of the VM are copied to the destination host. As the VM runs on the destination host, and encounters pages that it lacks, but remain present on the source host, the VM demand faults those pages over the network from the source host. This process is called “resume during page-in” or RDPI, and enables the system to guarantee transparent migration success, even for VMs having large working sets of memory pages which have not been pre-copied.
During this period of RDPI, the failure domain of the VM is extended because the VM's memory exists both on the source host and the destination host. For example, if the source host or the destination host crashes during RDPI, the VM needs to be terminated. The same is true when there is a network failure.